6 research outputs found
Reinforcement Learning for Solving Stochastic Vehicle Routing Problem
This study addresses a gap in the utilization of Reinforcement Learning (RL)
and Machine Learning (ML) techniques in solving the Stochastic Vehicle Routing
Problem (SVRP) that involves the challenging task of optimizing vehicle routes
under uncertain conditions. We propose a novel end-to-end framework that
comprehensively addresses the key sources of stochasticity in SVRP and utilizes
an RL agent with a simple yet effective architecture and a tailored training
method. Through comparative analysis, our proposed model demonstrates superior
performance compared to a widely adopted state-of-the-art metaheuristic,
achieving a significant 3.43% reduction in travel costs. Furthermore, the model
exhibits robustness across diverse SVRP settings, highlighting its adaptability
and ability to learn optimal routing strategies in varying environments. The
publicly available implementation of our framework serves as a valuable
resource for future research endeavors aimed at advancing RL-based solutions
for SVRP.Comment: 14 pages, accepted to ACML2
Reinforcement Learning for Solving Stochastic Vehicle Routing Problem with Time Windows
This paper introduces a reinforcement learning approach to optimize the
Stochastic Vehicle Routing Problem with Time Windows (SVRP), focusing on
reducing travel costs in goods delivery. We develop a novel SVRP formulation
that accounts for uncertain travel costs and demands, alongside specific
customer time windows. An attention-based neural network trained through
reinforcement learning is employed to minimize routing costs. Our approach
addresses a gap in SVRP research, which traditionally relies on heuristic
methods, by leveraging machine learning. The model outperforms the Ant-Colony
Optimization algorithm, achieving a 1.73% reduction in travel costs. It
uniquely integrates external information, demonstrating robustness in diverse
environments, making it a valuable benchmark for future SVRP studies and
industry application
AI for Porosity and Permeability Prediction from Geologic Core X-Ray Micro-Tomography
Geologic cores are rock samples that are extracted from deep under the ground
during the well drilling process. They are used for petroleum reservoirs'
performance characterization. Traditionally, physical studies of cores are
carried out by the means of manual time-consuming experiments. With the
development of deep learning, scientists actively started working on developing
machine-learning-based approaches to identify physical properties without any
manual experiments. Several previous works used machine learning to determine
the porosity and permeability of the rocks, but either method was inaccurate or
computationally expensive. We are proposing to use self-supervised pretraining
of the very small CNN-transformer-based model to predict the physical
properties of the rocks with high accuracy in a time-efficient manner. We show
that this technique prevents overfitting even for extremely small datasets.
Github: https://github.com/Shahbozjon/porosity-and-permeability-predictio
Analytics of Heterogeneous Breast Cancer Data Using Neuroevolution
https://ieeexplore.ieee.org/document/8632897Breast cancer prognostic modeling is difficult since it is governed by many diverse factors. Given the low median survival and large scale breast cancer data, which comes from high throughput technology, the accurate and reliable prognosis of breast cancer is becoming increasingly difficult. While accurate and timely prognosis may save many patients from going through painful and expensive treatments, it may also help oncologists in managing the disease more efficiently and effectively. Data analytics augmented by machine-learning algorithms have been proposed in past for breast cancer prognosis; and however, most of these could not perform well owing to the heterogeneous nature of available data and model interpretability related issues. A robust prognostic modeling approach is proposed here whereby a Pareto optimal set of deep neural networks (DNNs) exhibiting equally good performance metrics is obtained. The set of DNNs is initialized and their hyperparameters are optimized using the evolutionary algorithm, NSGAIII. The final DNN model is selected from the Pareto optimal set of many DNNs using a fuzzy inferencing approach. Contrary to using DNNs as the black box, the proposed scheme allows understanding how various performance metrics (such as accuracy, sensitivity, F1, and so on) change with changes in hyperparameters. This enhanced interpretability can be further used to improve or modify the behavior of DNNs. The heterogeneous breast cancer database requires preprocessing for better interpretation of categorical variables in order to improve prognosis from classifiers. Furthermore, we propose to use a neural network-based entity-embedding method for categorical features with high cardinality. This approach can provide a vector representation of categorical features in multidimensional space with enhanced interpretability. It is shown with evidence that DNNs optimized using evolutionary algorithms exhibit improved performance over other classifiers mentioned in this paper
Backward algorithm and abstract graphs in zero-sum games
With the beginning of the computer age, solving many problems in game theory has
become easier. However, there is a whole class of well-known problems such as chess,
checkers, go and so on, the methods of solving which use brute force technique to find
solutions of the game. This technique requires analysis of all states of the game and it has
the exponential complexity of running the algorithm due to which many games cannot
be solved on modern computers. Considered class of games include zero-sum games with
perfect information described in discrete space. For such problems, there is no smooth
solution that would allow solving problems without going through all the states of the
game. This work proposes a new algorithm for finding solutions to such problems. The
algorithm uses a new data structure, called abstract graphs and backwards analysis to
find solutions. The algorithm still has the exponential complexity of the analysis, however,
finding a solution does not require going through all the possible states of the game, which
reduces the complexity of analysis. For a clear example, the algorithm was used on a
tic-tac-toe game, for which brute force technique requires analysis of around 15k states,
while the Backwards algorithm analyzed just 5 states to find all existing solutions. In the
future, this study can be continued for a deeper study of the mathematical properties of
the algorithm and to use it on games such as chess and go
Robust Reinforcement Learning on Graphs for Logistics optimization
Logistics optimization nowadays is becoming one of the hottest areas in the
AI community. In the past year, significant advancements in the domain were
achieved by representing the problem in a form of graph. Another promising area
of research was to apply reinforcement learning algorithms to the above task.
In our work, we made advantage of using both approaches and apply reinforcement
learning on a graph. To do that, we have analyzed the most recent results in
both fields and selected SOTA algorithms both from graph neural networks and
reinforcement learning. Then, we combined selected models on the problem of
AMOD systems optimization for the transportation network of New York city. Our
team compared three algorithms - GAT, Pro-CNN and PTDNet - to bring to the fore
the important nodes on a graph representation. Finally, we achieved SOTA
results on AMOD systems optimization problem employing PTDNet with GNN and
training them in reinforcement fashion.
Keywords: Graph Neural Network (GNN), Logistics optimization, Reinforcement
LearningComment: Keywords: Graph Neural Network (GNN), Logistics optimization,
Reinforcement Learnin